Actively controlling coolant-cooled cold plate configuration

ABSTRACT

A method is provided to facilitate active control of thermal and fluid dynamic performance of a coolant-cooled cold plate. The method includes: monitoring a variable associated with at least one of the coolant-cooled cold plate or one or more electronic components being cooled by the cold plate; and dynamically varying, based on the monitored variable, a physical configuration of the cold plate. By dynamically varying the physical configuration, the thermal and fluid dynamic performance of the cold plate are adjusted to, for example, optimally cool the one or more electronic components, and at the same time, reduce cooling power consumption used in cooling the electronic component(s). The physical configuration can be adjusted by providing one or more adjustable plates within the coolant-cooled cold plate, the positioning of which may be adjusted based on the monitored variable.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH

This invention was made with Government support under Contract No.DE-EE0002894, awarded by the Department of Energy (DOE). Accordingly,the U.S. Government has certain rights in the invention.

BACKGROUND

The power dissipation of integrated circuit chips, and the modulescontaining the chips, continues to increase in order to achieveincreases in processor performance. This trend poses cooling challengesat the module and system levels.

In many large server applications, processors along with theirassociated electronics (e.g., memory, disk drives, power supplies, etc.)are packaged in removable drawer configurations stacked within anelectronics rack or frame comprising information technology (IT)equipment. In other cases, the electronics may be in fixed locationswithin the rack or frame. Typically, the components are cooled by airmoving in parallel airflow paths, usually front-to-back, impelled by oneor more air moving devices (e.g., fans or blowers). In some cases it maybe possible to handle increased power dissipation within a single draweror subsystem by providing greater airflow, for example, through the useof a more powerful air moving device or by increasing the rotationalspeed (i.e., RPMs) of an existing air moving device. However, thisapproach is becoming problematic, particularly in the context of acomputer center installation (i.e., data center).

The sensible heat load carried by the air exiting the rack is stressingthe capability of the room air-conditioning to effectively handle theload. This is especially true for large installations with “serverfarms” or large banks of computer racks located close together. In suchinstallations, liquid-cooling is an attractive technology to manage thehigher heat fluxes. The liquid absorbs the heat dissipated by thecomponents/modules in an efficient manner. Typically, the heat isultimately transferred from the liquid to an outside environment,whether air or other liquid.

BRIEF SUMMARY

The shortcomings of the prior art are overcome and additional advantagesare provided through the provision of a method comprising: monitoring avariable associated with at least one of a coolant-cooled cold plate oran electronic component being cooled by the coolant-cooled cold plate;and dynamically varying, based on the monitored variable, a physicalconfiguration of the coolant-cooled cold plate, wherein the dynamicallyvarying the physical configuration alters at least one of thermal orfluid dynamic performance of the coolant-cooled cold plate cooling theelectronic component.

In another aspect, a method is provided which includes: monitoring avariable associated with at least one of a coolant-cooled cold plate oran electronic component being cooled by the coolant-cooled cold plate;and dynamically varying, based on the monitored variable, a physicalconfiguration of the coolant-cooled cold plate, the dynamically varyingcomprising dynamically reconfiguring the physical configuration of thecoolant-cooled cold plate by automatically adjusting at least oneadjustable plate within the coolant-cooled cold plate, wherein thedynamically varying the physical configuration alters at least one ofthermal or fluid dynamic performance of the coolant-cooled cold platecooling the electronic component.

In a further aspect, a method is provided which includes: providing acoolant-cooled cold plate configured to couple to at least oneelectronic component to be cooled, the coolant-cooled cold platecomprising at least one coolant-carrying channel, and at least oneadjustable plate for adjusting a physical configuration of thecoolant-cooled cold plate, wherein adjustment of the at least oneadjustable plate reconfigures the coolant-cooled cold plate and altersat least one of thermal or fluid dynamic performance of thecoolant-cooled cold plate; and providing a controller to activelycontrol positioning of the at least one adjustable plate within thecoolant-cooled cold plate based on a monitored variable associated withat least one of the coolant-cooled cold plate or the at least oneelectronic component to be cooled by the coolant-cooled cold plate.

Additional features and advantages are realized through the techniquesof the present invention. Other embodiments and aspects of the inventionare described in detail herein and are considered a part of the claimedinvention.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

One or more aspects of the present invention are particularly pointedout and distinctly claimed as examples in the claims at the conclusionof the specification. The foregoing and other objects, features, andadvantages of the invention are apparent from the following detaileddescription taken in conjunction with the accompanying drawings inwhich:

FIG. 1 depicts one embodiment of a conventional raised floor layout ofan air-cooled computer installation;

FIG. 2 is a front elevational view of one embodiment of a liquid-cooledelectronics rack comprising multiple electronic systems to be cooled viaa cooling apparatus, in accordance with one or more aspects of thepresent invention;

FIG. 3 is a schematic of an electronic system of an electronics rack andone approach to liquid-cooling of an electronic component with theelectronic system, wherein the electronic component is indirectlyliquid-cooled by system coolant provided by one or more modular coolingunits disposed within the electronics rack, in accordance with one ormore aspects of the present invention;

FIG. 4 is a schematic of one embodiment of a modular cooling unit for aliquid-cooled electronics rack such as illustrated in FIG. 2, inaccordance with one or more aspects of the present invention;

FIG. 5 is a plan view of one embodiment of an electronic system layoutillustrating an air and liquid-cooling approach for cooling electroniccomponents of the electronic system, in accordance with one or moreaspects of the present invention;

FIG. 6 is a plan view of another embodiment of an electronic systemlayout for a liquid-cooled electronics rack, and illustrating multiplecoolant-cooled cold plates and multiple coolant-cooled cold railscoupled in fluid communication, in accordance with one or more aspectsof the present invention;

FIG. 7A is a cross-sectional elevational view of one embodiment of acooling apparatus comprising a coolant-cooled cold plate with anadjustable physical characteristic, in accordance with one or moreaspects of the present invention;

FIG. 7B is a plan view of one embodiment of the coolant-cooled coldplate of FIG. 7A, in accordance with one or more aspects of the presentinvention;

FIG. 8A is a cross-sectional elevational view of the coolant-cooled coldplate of FIGS. 7A & 7B, with the adjustable mid-plate thereof shown at afirst stage, in accordance with one or more aspects of the presentinvention;

FIG. 8B is a cross-sectional elevational view of the coolant-cooled coldplate of FIGS. 7A & 7B, with the adjustable mid-plate thereof shown at asecond stage, in accordance with one or more aspects of the presentinvention;

FIG. 8C is a cross-sectional elevational view of the coolant-cooled coldplate of FIGS. 7A & 7B, with the adjustable mid-plate thereof shown at athird stage, in accordance with one or more aspects of the presentinvention;

FIG. 9 depicts one embodiment of a process for actively controllingphysical configuration of a coolant-cooled cold plate, in accordancewith one or more aspects of the present invention;

FIG. 10 is a graph illustrating a heat transfer benefit by dynamicallymodifying cross-sectional flow area through one or more channels of acoolant-cooled cold plate, in accordance with one or more aspects of thepresent invention;

FIG. 11 is a graph illustrating change in Reynolds Number (Re), and theeffect thereof on the heat transfer coefficient within a coolant-cooledcold plate, in accordance with one or more aspects of the presentinvention;

FIG. 12A depicts an alternate embodiment of a cooling apparatuscomprising a coolant-cooled cold plate with an adjustable physicalcharacteristic, and illustrating the cold plate in a first stage, inaccordance with one or more aspects of the present invention;

FIG. 12B is a plan view of the coolant-cooled cold plate of FIG. 12A, inaccordance with one or more aspects of the present invention;

FIG. 12C is a cross-sectional elevational view of the coolant-cooledcold plate of FIGS. 12A & 12B, with the cold plate shown in a secondstage, wherein the cold plate is reconfigured to reduce the availablecross-sectional flow area through the cold plate, in accordance with oneor more aspects of the present invention;

FIG. 12D is a plan view of the coolant-cooled cold plate of FIG. 12C,shown in the second stage, in accordance with one or more aspects of thepresent invention;

FIG. 12E is a cross-sectional elevational view of the coolant-cooledcold plate of FIGS. 12A-12D, with the cold plate shown in a third stage,wherein the cold plate is reconfigured to further reduce cross-sectionalflow area through the cold plate, in accordance with one or more aspectsof the present invention;

FIG. 12F is a plan view of the coolant-cooled cold plate of FIG. 12E,shown in the third stage, in accordance with one or more aspects of thepresent invention;

FIG. 13A depicts another embodiment of a cooling apparatus comprising acoolant-cooled cold plate with an adjustable physical characteristic, inaccordance with one or more aspects of the present invention;

FIG. 13B is a plan view of the coolant-cooled cold plate of FIG. 13A, inaccordance with one or more aspects of the present invention;

FIG. 13C is a cross-sectional elevational view of the coolant-cooledcold plate of FIGS. 13A & 13B, with the cold plate shown in a firststage, wherein the adjustable physical characteristic is configured topresent a maximum available cross-sectional flow area through the coldplate, in accordance with one or more aspects of the present invention;

FIG. 13D is a cross-sectional elevational view of the coolant-cooledcold plate of FIGS. 13A-13C, with the cold plate shown in a secondstage, wherein the adjustable physical characteristic is reconfigured toreduce the available cross-sectional flow area through the cold plate,in accordance with one or more aspects of the present invention;

FIG. 13E is a cross-sectional elevational view of the coolant-cooledcold plate of FIGS. 13A-13D, with the cold plate shown in a third stage,wherein the adjustable physical characteristic is reconfigured tofurther reduce the available cross-sectional flow area through the coldplate, in accordance with one or more aspects of the present invention;

FIG. 14A depicts a further embodiment of a cooling apparatus comprisinga coolant-cooled cold plate with an adjustable physical characteristic,wherein the cold plate comprises an adjustable jet orifice plate, inaccordance with one or more aspects of the present invention;

FIG. 14B is a plan view of the coolant-cooled cold plate of FIG. 14A, inaccordance with one or more aspects of the present invention;

FIG. 14C is a cross-sectional elevational view of the coolant-cooledcold plate of FIGS. 14A & 14B, with the jet orifice plate shown in afirst stage, where jet impingement height is highest, in accordance withone or more aspects of the present invention;

FIG. 14D is a cross-sectional elevational view of the coolant-cooledcold plate of FIGS. 14A-14C, with the jet orifice plate shown in asecond stage, wherein jet impingement height is reduced from the firststage, in accordance with one or more aspects of the present invention;

FIG. 14E is a cross-sectional elevational view of the coolant-cooledcold plate of FIGS. 14A-14E, with the jet orifice plate shown in a thirdstage, wherein jet impingement height is reduced from the second stage,in accordance with one or more aspects of the present invention;

FIG. 15A depicts another embodiment of a cooling apparatus comprising acoolant-cooled cold plate with a physical configuration controlled by anadjustable piezoelectric mid-plate, the mid-plate being shown deflectedaway from the channel base to increase available cross-section flow areathrough the cold plate, in accordance with one or more aspects of thepresent invention;

FIG. 15B is a cross-sectional elevational view of the coolant-cooledcold plate of FIG. 15A, with the adjustable piezoelectric mid-plateundeflected, in accordance with one or more aspects of the presentinvention;

FIG. 15C is a cross-sectional elevational view of the coolant-cooledcold plate of FIGS. 15A & 15B, with the adjustable piezoelectricmid-plate deflected towards the channel base to present a smalleravailable cross-sectional flow area through the cold plate, inaccordance with one or more aspects of the present invention; and

FIG. 16 depicts one embodiment of a computer program productincorporating one or more aspects of the present invention, inaccordance with one or more aspects of the present invention.

DETAILED DESCRIPTION

As used herein, the terms “electronics rack”, “rack-mounted electronicequipment”, and “rack unit” are used interchangeably, and unlessotherwise specified include any housing, frame, rack, compartment, bladeserver system, etc., having one or more heat-generating components of acomputer system, electronic system, or information technology equipment,and may be, for example, a stand alone computer processor having high-,mid- or low-end processing capability. In one embodiment, an electronicsrack may comprise a portion of an electronic system, a single electronicsystem, or multiple electronic systems, for example, in one or moresub-housings, blades, books, drawers, nodes, compartments, etc., havingone or more heat-generating electronic components disposed therein. Anelectronic system(s) within an electronics rack may be movable or fixed,relative to the electronics rack, with rack-mounted electronic drawersand blades of a blade center system being two examples of electronicsystems (or subsystems) of an electronics rack to be cooled.

“Electronic component” refers to any heat generating electroniccomponent of, for example, a computer system or other electronics unitrequiring cooling. By way of example, an electronic component maycomprise one or more integrated circuit dies and/or other electronicdevices to be cooled, including one or more processor dies, memory diesor memory support dies. As a further example, the electronic componentmay comprise one or more bare dies or one or more packaged dies disposedon a common carrier. Further, unless otherwise specified herein, theterms “coolant-cooled cold plate”, “coolant-cooled cold rail”, or“coolant-cooled structure” refer to a thermally conductive structurehaving a one or more channels or passageways formed therein for flowingof coolant therethrough. In one example, the coolant is a liquidcoolant.

As used herein, a “liquid-to-liquid heat exchanger” may comprise, forexample, two or more coolant flow paths, formed of thermally conductivetubing (such as copper or other tubing) in thermal or mechanical contactwith each other. Size, configuration and construction of theliquid-to-liquid heat exchanger can vary without departing from thescope of the invention disclosed herein. Further, “data center” refersto a computer installation containing one or more electronics racks tobe cooled. As a specific example, a data center may include one or morerows of rack-mounted computing units, such as server units.

One example of facility coolant and system coolant is water. However,the concepts disclosed herein are readily adapted to use with othertypes of coolant on the facility side and/or on the system side. Forexample, one or more of these coolants may comprise a brine, adielectric liquid, a fluorocarbon liquid, a liquid metal, or othersimilar coolant, or a refrigerant, while still maintaining theadvantages and unique features of the present invention.

Reference is made below to the drawings, which are not drawn to scalefor ease of understanding of the various aspects of the presentinvention, wherein the same reference numbers used throughout differentfigures designate the same or similar components.

As shown in FIG. 1, in a raised floor layout of an air-cooled datacenter 100 typical in the prior art, multiple electronics racks 110 aredisposed in one or more rows. A computer installation such as depictedin FIG. 1 may house several hundred, or even several thousandmicroprocessors. In the arrangement of FIG. 1, chilled air enters thecomputer room via floor vents from a supply air plenum 145 definedbetween the raised floor 140 and a base or sub-floor 165 of the room.Cooled air is taken in through louvered covers at air inlet sides 120 ofthe electronics racks and expelled through the backs, i.e., air outletsides 130, of the electronics racks. Each electronics rack 110 may haveone or more air-moving devices (e.g., fans or blowers) to provide forcedinlet-to-outlet air flow to cool the electronic components within thedrawer(s) of the rack. The supply air plenum 145 provides conditionedand cooled air to the air-inlet sides of the electronics racks viaperforated floor tiles 160 disposed in a “cold” aisle of the computerinstallation. The conditioned and cooled air is supplied to plenum 145by one or more air conditioning units 150, also disposed within datacenter 100. Room air is taken into each air conditioning unit 150 nearan upper portion thereof. This room air may comprise (in part) exhaustedair from the “hot” aisles of the computer installation defined byopposing air outlet sides 130 of the electronics racks 110.

FIG. 2 depicts one embodiment of a liquid-cooled electronics rack 200comprising a cooling apparatus. In one embodiment, liquid-cooledelectronics rack 200 comprises a plurality of electronic systems 210,which may be processor or server nodes (in one embodiment). A bulk powerassembly 220 is disposed at an upper portion of liquid-cooledelectronics rack 200, and two modular cooling units (MCUs) 230 arepositioned in a lower portion of the liquid-cooled electronics rack forproviding system coolant to the electronic systems. In the embodimentsdescribed herein, the system coolant is assumed to be water or anaqueous-based solution, by way of example only.

In addition to MCUs 230, the cooling apparatus depicted includes asystem coolant supply manifold 231, a system coolant return manifold232, and manifold-to-node fluid connect hoses 233 coupling systemcoolant supply manifold 231 to electronic subsystems 210 (for example,to cold plates or liquid-cooled vapor condensers (see FIGS. 6A-9B)disposed within the systems) and node-to-manifold fluid connect hoses234 coupling the individual electronic systems 210 to system coolantreturn manifold 232. Each MCU 230 is in fluid communication with systemcoolant supply manifold 231 via a respective system coolant supply hose235, and each MCU 230 is in fluid communication with system coolantreturn manifold 232 via a respective system coolant return hose 236.

Heat load of the electronic systems 210 is transferred from the systemcoolant to cooler facility coolant within the MCUs 230 provided viafacility coolant supply line 240 and facility coolant return line 241disposed, in the illustrated embodiment, in the space between raisedfloor 145 and base floor 165.

FIG. 3 schematically illustrates one cooling approach using the coolingapparatus of FIG. 2, wherein a liquid-cooled cold plate 300 is showncoupled to an electronic component 301 of an electronic system 210within the liquid-cooled electronics rack 200. Heat is removed fromelectronic component 301 via system coolant circulating via pump 320through liquid-cooled cold plate 300 within the system coolant loopdefined, in part, by liquid-to-liquid heat exchanger 321 of modularcooling unit 230, hoses 235, 236 and cold plate 300. The system coolantloop and modular cooling unit are designed to provide coolant of acontrolled temperature and pressure, as well as controlled chemistry andcleanliness to the electronic systems. Furthermore, the system coolantis physically separate from the less controlled facility coolant inlines 240, 241, to which heat is ultimately transferred.

FIG. 4 depicts one detailed embodiment of a modular cooling unit 230. Asshown in FIG. 4, modular cooling unit 230 includes a facility coolantloop, wherein building chilled, facility coolant is provided (via lines240, 241) and passed through a control valve 420 driven by a motor 425.Valve 420 determines an amount of facility coolant to be passed throughheat exchanger 321, with a portion of the facility coolant possiblybeing returned directly via a bypass orifice 435. The modular coolingunit further includes a system coolant loop with a reservoir tank 440from which system coolant is pumped, either by pump 450 or pump 451,into liquid-to-liquid heat exchanger 321 for conditioning and outputthereof, as cooled system coolant to the electronics rack to be cooled.Each modular cooling unit is coupled to the system supply manifold andsystem return manifold of the liquid-cooled electronics rack via thesystem coolant supply hose 235 and system coolant return hose 236,respectively.

FIG. 5 depicts another cooling approach, illustrating one embodiment ofan electronic system 210 component layout wherein one or more air movingdevices 511 provide forced air flow 515 in normal operating mode to coolmultiple electronic components 512 within electronic system 210. Coolair is taken in through a front 531 and exhausted out a back 533 of thedrawer. The multiple components to be cooled include multiple processormodules to which liquid-cooled cold plates 520 are coupled, as well asmultiple arrays of memory modules 530 (e.g., dual in-line memory modules(DIMMs)) and multiple rows of memory support modules 532 (e.g., DIMMcontrol modules) to which air-cooled heat sinks may be coupled. In theembodiment illustrated, memory modules 530 and the memory supportmodules 532 are partially arrayed near front 531 of electronic system210, and partially arrayed near back 533 of electronic system 210. Also,in the embodiment of FIG. 5, memory modules 530 and the memory supportmodules 532 are cooled by air flow 515 across the electronics system.

The illustrated cooling apparatus further includes multiplecoolant-carrying tubes connected to and in fluid communication withliquid-cooled cold plates 520. The coolant-carrying tubes comprise setsof coolant-carrying tubes, with each set including (for example) acoolant supply tube 540, a bridge tube 541 and a coolant return tube542. In this example, each set of tubes provides liquid-coolant to aseries-connected pair of cold plates 520 (coupled to a pair of processormodules). Coolant flows into a first cold plate of each pair via thecoolant supply tube 540 and from the first cold plate to a second coldplate of the pair via bridge tube or line 541, which may or may not bethermally conductive. From the second cold plate of the pair, coolant isreturned through the respective coolant return tube 542.

FIG. 6 illustrates another embodiment of a cooled electronic system 210′component layout, wherein one or more air-moving devices 600 provideforced air flow 601 to cool multiple components 610 within electronicsystem 210′. Cool air is taken in through a front 602 and exhausted outa back 603 of the electronic system (or drawer). The multiple componentsto be cooled include, for example, multiple processor modules to whichcoolant-cooled cold plates 620 (of the coolant-based cooling apparatus)are coupled, as well as multiple arrays 631, 632 of electronics cards630 (e.g., memory modules such as dual in-line memory modules (DIMMs)),which are to be thermally coupled to one or more coolant-cooled coldrails 625. As used herein “thermally coupled” refers to a physicalthermal transport path being established between components, forexample, between an electronics card and a coolant-cooled cold rail forthe conduction of heat from one to the other.

The illustrated liquid-based cooling approach further includes multiplecoolant-carrying tubes connecting in fluid communication coolant-cooledcold plates 620 and coolant-cooled cold rails 625. Thesecoolant-carrying tubes comprise (for example), a coolant supply tube640, multiple bridge tubes 641, and a coolant return tube 642. In theembodiment illustrated, bridge tubes 641 connect one coolant-cooled coldrail 625 in series between the two coolant-cooled cold plates 620, andconnect in parallel two additional coolant-cooled cold rails 625 betweenthe second coolant-cooled cold plate 620 and the coolant return tube642. Note that this configuration is provided by way of example only.The concepts disclosed herein may be readily adapted to use with variousconfigurations of cooled electronic system layouts. Note also, that asdepicted herein, the coolant-cooled cold rails are elongate, thermallyconductive structures comprising one or more channels through whichcoolant passes, for example, via one or more tubes extending through thestructures. The coolant-cooled cold rails are disposed, in theembodiment illustrated, at the ends of the two arrays (or banks) 631,632 of electronics cards 630, and multiple thermal spreaders areprovided coupling in thermal communication electronics cards 630 andcoolant-cooled cold rails 625.

In many cases, one or more electronic components (e.g., processors) inone or more electronic systems (e.g., servers) of a rack unit might runat different states dissipating different amounts of power.Additionally, in another case of uneven power dissipation, electronicsystems in an electronics rack might have different numbers ofelectronic components, such as processors, dissipating different amountsof power. In such cases, the liquid-coolant flow rate within thecoolant-cooled rack is typically decided based upon the hottestcomponent(s) in the system, or in the rack of electronic systems. Thehottest component is optimally cooled, but other components may beover-cooled, consuming more cooling power than necessary.

Advantageously, disclosed herein are cooling methods and coolingapparatuses with various coolant-cooled cold plate designs for activelycontrolling a physical characteristic of the cold plates to dynamicallyalter at least one of thermal or fluid dynamic performance of the coldplate when cooling one or more electronic components. The monitoredvariable could comprise, as one example, a temperature associated withat least one of the coolant-cooled cold plate, or the one or moreelectronic components being cooled by the coolant-cooled cold plate.Dynamically altering the thermal and hydrodynamic performance of thecold plate can be employed to optimally cool the electronic component,while at the same time reducing cooling power consumption.

Disclosed herein are various embodiments of cold plates for activelycontrolling cooling of electronic components so as to, for example,optimally cool electronic components in a coolant-cooled electronicsrack comprising multiple electronic systems with multipleheat-generating electronic components being cooled by multiplecoolant-cooled cold plates. Actively controlling configuration of a coldplate may, in one embodiment, include dynamically reconfiguring acoolant flow cross-section through the cold plate based on the monitoredvariable. In an alternate configuration, actively controlling cold plateconfiguration might include dynamically reconfiguring the cold plate toadjust height of a jet orifice plate within the cold plate relative to atarget surface, for example, of the cold plate. Numerous implementationsof the active control concept disclosed herein are described below withreference to FIGS. 7A-15C.

Referring first to FIGS. 7A & 7B, one embodiment of a cooling apparatus700, in accordance with one or more aspects of the present invention, isillustrated. Cooling apparatus 700 includes a coolant-cooled cold plate710 and a controller 730 for actively controlling configuration, andthus a physical characteristic, of the coolant-cooled cold plate. Thecoolant-cooled cold plate 710 includes, in this embodiment, a pluralityof parallel-extending, coolant-carrying channels 711 separated bychannel walls 712. A coolant flow cross-section through a singlecoolant-carrying channel of the plurality of coolant-carrying channels711 is determined by a width ‘W’ and a height ‘H’ of thecoolant-carrying channel. As described herein, by adjusting the height‘H’ of the coolant flow channels, the coolant flow cross-section throughthe channels, as well as through the coolant-cooled cold plate, can beautomatically, dynamically controlled. Automatically controlling thecoolant flow cross-sections through the channels, and thus through thecold plate, is one example of actively controlling (or dynamicallyvarying), based on a monitored variable, a physical configuration of thecoolant-cooled cold plate to alter at least one of the thermal or fluiddynamic performance of the coolant-cooled cold plate.

In the depicted embodiment, an adjustable mid-plate 715 is provided,which is controlled by controller 730 to adjust the height ‘H’ of theparallel, coolant-carrying channels 711 relative to a base plate 713 ofcoolant-cooled cold plate 710. In this example, adjustable mid-plate 715is magnetically adjustable, with permanent magnets 716 being affixed toor integrated within adjustable mid-plate 715, and controllable magnets717 associated with an upper plate 718 of coolant-cooled cold plate 710.By controlling the magnetic force applied by controllable magnets 717,including polarity of the magnets, the adjustable mid-plate 715 can bemoved to adjust the effective channel height ‘H’ of the plurality ofparallel, coolant-carrying channels 711. As noted, in thisimplementation, the adjustable channel height is a physicalcharacteristic of the coolant-cooled cold plate that is being changedvia control of the magnetic force being applied to the adjustablemid-plate 715. Hard stops 714 are provided, for example, at a desiredminimum channel height ‘H’.

Coolant flows into the parallel, coolant-carrying channels 711 via oneor more coolant inlets 720 and egresses via one or more coolant outlets721. Note that the space 719 between adjustable mid-plate 715 and upperplate 718 of the coolant-cooled cold plate may be filled with coolant,for example, to facilitate heat transfer to upper plate 718 of the coldplate. In such a case, however, the coolant flow is through (orprincipally through) the plurality of parallel, coolant-carryingchannels 711.

Controller 730 may be within a common electronic subsystem containingthe coolant-cooled cold plate 710, or could be disposed in anotherelectronic system within the same electronics rack, or even remote fromthe electronics rack containing the coolant-cooled cold plate (with theadjustable physical characteristic). The controller implements a controlprocess for automatically controlling the position of adjustablemid-plate 715, and thus for controlling (in this example) coolant flowcross-section through the plurality of coolant-carrying channels, andtherefore through the cold plate. Thus, controller 720 actively controlsa physical characteristic of the coolant-cooled cold plate by changingcold plate configuration, and this active control is to, for instance,optimally cool the associated electronic components being cooled by thecold plate, and at the same time, reduce cooling power consumption. Inone embodiment, the adjustable mid-plate is adjusted by the controllerto reconfigure the effective channel height based on, for example, heatdissipated by the associated electronic component(s) being cooled,and/or the coolant flow rate through the coolant-cooled cold plate.

The adjustable mid-plate 715 may contain grooves or slots correspondingto each channel wall 712 for movement normal to the parallel-extending,coolant-carrying channel lengths. As noted, polarity of the controllablemagnets 717 associated with the upper plate 718 of the cold plate can betoggled, and the magnetic strength can be varied, to attract or repelthe adjustable mid-plate as desired. Thus, by controlling the magneticforce between the adjustable mid-plate and the upper plate, theeffective channel height can be controlled and adjusted.

FIGS. 8A-8C depict three different stages of mid-plate adjustment.

In FIG. 8A, a stage 1 position is illustrated, wherein the controllablemagnets have opposite polarity to the permanent magnets affixed to theadjustable mid-plate, such that the mid-plate is attracted to the topplate. The result is a maximum channel height ‘H’ for coolant flowthrough the coolant-cooled cold plate. This in turn results in arelatively lower Reynolds Number (Re) for the same coolant mass flowrate through the cold plate.

FIG. 8B depicts a stage 2 configuration, wherein the magnetic components717 on upper plate 718 could have similar polarity, or partially similarand partially opposite polarity, as the permanent magnets 716 onadjustable mid-plate 715. The magnetic strength of the magneticcomponents 717 could be varied as well so that the adjustable mid-platestays at an intermediate position, such as illustrated in FIG. 8B. Inthis position, the effective channel height ‘H’ is lower, and thus, thecoolant flow cross-section through the respective channels and throughthe cold plate is smaller, resulting in a relatively higher ReynoldsNumber for the same coolant mass flow rate (that is, compared with thatof stage 1).

In FIG. 8C, a stage 3 is illustrated, wherein the adjustable magneticcomponents 717 on upper plate 718 have similar polarity as the permanentmagnets 716 on adjustable mid-plate 715, so that the plates repel. Themagnetic strength of the magnetic components 717 on upper plate 718 canbe adjusted so that the adjustable mid-plate is farthest away from theupper plate, resulting in a smallest possible effective channel height‘H’. In this implementation, the adjustable mid-plate 715 may rest onhard stops 714 disposed, for example, on the inner side walls of thecoolant-cooled plate. As illustrated, in stage 3, a smaller coolant flowcross-section is presented through the plurality of parallel coolantchannels, and thus, through the cold plate (that is, compared with thatof stage 2). This smaller flow cross-section results in relativelyhigher Reynolds Number for the same coolant mass flow rate.

It should be noted that the convective heat transfer rates are higherfor larger Reynolds Number flows. Thus, for a given coolant mass flowrate through the coolant-cooled cold plate, stage 3 offers a higher heattransfer rate compared to stage 2, and stage 2 offers a higher heattransfer rate compared to stage 1.

FIG. 9 depicts one embodiment of a process for actively controllingoperation of a coolant-cooled cold plate such as depicted in FIGS. 7A &7B, between the three stages of FIGS. 8A-8C. Table 1 below containsnomenclature for the following discussion of FIGS. 9-11.

TABLE 1 SYMBOL DEFINITION A — Channel cross-sectional area A₁ — Channelcross-sectional area for relatively taller channels A₂ — Channelcross-sectional area for relatively shorter channels D_(h) — HydraulicDiameter of the channels D_(h,1) — Hydraulic Diameter of the relativelytaller channels D_(h,2) — Hydraulic Diameter of the relatively shorterchannels h — Convective heat transfer coefficient h₁ — Convective heattransfer coefficient in the developed flow regime for the relativelytaller channels h₂ — Convective heat transfer coefficient in thedeveloped flow regime for the relatively shorter channels k — Thermalconductivity of the coolant l_(e) — Thermal development length l_(e,1) —Thermal development length for relatively taller channels l_(e,2) —Thermal development length for relatively shorter channels m — Coolantmass flow rate Nu — Nusselt Number P — Perimeter of the channelcross-section P₁ — Perimeter of the channel cross-section for relativelytaller channels P₂ — Perimeter of the channel cross-section forrelatively shorter channels Pr — Prandtl Number of the coolant (eg.Water 7, air 0.7, etc.) Re — Reynolds Number Re₁ — Reynolds Number forrelatively taller channels Re₂ — Reynolds Number for relatively shorterchannels T_(j) — CPU Junction Temperature (that is, Device Temperature)T_(j,Cr1) — Higher/Upper Critical Temperature Value T_(j,spec1) — UpperTemperature specification T_(j,spec2) — Lower Temperature specificationT_(j,Cr2) — Lower Critical Temperature value (Note: T_(j,Cr1) >T_(j,spec1) > T_(j,spec2) > T_(j,Cr2)) μ — Coolant viscosity

In the example of FIG. 9, active control processing is based onelectronic component junction temperature (T_(j)). In one example, theelectronic component may comprise one or more processors, and thejunction temperature is employed as a measure of the heat dissipated bythe one or more processors being cooled by the coolant-cooled coldplate.

Upon entering the control loop, processing determines whether activecontrol is enabled 900. If “no”, then the control loop is exited, andthe system continues with normal operation 902. In this case, thecoolant-cooled cold plate might remain in a previously setconfiguration. If, however, active control is enabled, then processingdetermines whether the junction temperature (T_(j)) is less than a firstcritical value (T_(j,Cr1)) 904. If “no”, then processing determineswhether the coolant-cooled cold plate is operating at stage 3 906, andif “yes”, a “Low Coolant Flow” warning is issued, and coolant flow ratethrough the cold plate is increased 910. The controller then waits atime interval t 912 before again checking whether active cooling isenabled 900. If the junction temperature is greater than the firstcritical value, and the coolant-cooled cold plate is not operating atstage 3, the controller switches the cold plate to stage 3 operation914, and then waits time interval t 912 before returning to checkwhether active control is enabled.

If the junction temperature (T_(j)) is less than the first criticaltemperature value (T_(j,Cr1)), processing determines whether thejunction temperature (T_(j)) is less than a first specified junctiontemperature (T_(j,spec1)) 916, wherein the first specified junctiontemperature (T_(j,spec1)) and a second specified junction temperature(T_(j,spec2)) define a desired, specified operating range for themonitored junction temperature (T_(j)). If the junction temperature(T_(j)) is not less than the first specified junction temperature(T_(j,spec1)), processing determines whether the cold plate is operatingat stage 1 918. If “yes”, then the controller switches the cold plate tostage 2 920, and waits time interval t 912 before again cycling throughthe control loop by determining whether active mode is enabled 900. If“no”, that is, if the cold plate is not operating at stage 1, processingdetermines whether the cold plate is operating at stage 2 922. If “yes”,then processing switches the cold plate to stage 3 924, and waits timeinterval t 912 before determining whether active control is enabled 900.If the cold plate is not operating at stage 2 922, processing determineswhether the cold plate is operating at stage 3 926. If “yes”, then a“Low Coolant Flow” warning is issued, and steps are taken to increasethe coolant flow rate 932. Processing then waits time interval t 912before returning to determine whether active control is enabled 900. Ifthe cold plate is not operating at stage 3 926, then processingswitching the cold plate to stage 3 928, and waits time interval t 912before determining whether active control is enabled 900.

If junction temperature (T_(j)) is less than the first specifiedjunction temperature (T_(j,spec1)), processing determines whetherjunction temperature is less than the second specified junctiontemperature (T_(j,spec2)) 934. If “no”, then processing determineswhether the cold plate is operating at stage 3 936. If “yes”, then thecold plate is switched to stage 2 938, and the system waits timeinterval t 912 before again checking whether the active control isenabled 900. If the cold plate is not operating at stage 3, thenprocessing determines whether the cold plate is operating at stage 2940. If “yes”, then the controller switches the cold plate to stage 1942, and waits time interval t 912 before again checking to determinewhether active control is enabled 900. If the cold plate is notoperating at stage 2 940, then processing determines whether the coldplate is operating at stage 1 944. If “yes”, then a “High Coolant Flow”warning is issued, and steps are taken to decrease the coolant flow rate950. Processing then waits time interval t 912 before determiningwhether active control is enabled 900. If the cold plate is notoperating at stage 1 944, then processing switches the cold plate tostage 1 946, and the system waits time interval t 912 before checkingwhether active mode is enabled, and repeating the control loop of FIG.9.

If junction temperature (T_(j)) is less than the second specifiedjunction temperature (T_(j,spec2)) 934, then processing determineswhether the junction temperature is less than a second critical junctiontemperature (T_(j,Cr2)) 952, wherein the second critical temperaturevalue (T_(j,Cr2)) is less than the second specified junction temperature(T_(j,spec2)). If “no”, then processing determines whether the coldplate is operating at stage 3 954, and if “yes”, the cold plate isswitched to stage 2 956, and processing waits time interval t 912 beforechecking if active control is enabled 900. If the cold plate is notoperating at stage 3 954, then processing determines whether the coldplate is operating at stage 2 958. If “yes”, the cold plate is switchedto stage 1 960, and the system waits time interval t 912 before checkingif active control is enabled 900. If the cold plate is not operating atstage 2 958, processing determines whether the cold plate is running atstage 1 962. If “yes”, then a “High Coolant Flow” warning is issued 966,and coolant flow rate is decreased 968, after which processing waitstime interval t 912 before determining whether active control is enabled900, to repeat the process. If the cold plate is not operating at stage1 962, then processing switches the cold plate to stage 1 964, and waitsfor time interval t 912 before checking if active control is enabled900.

If the junction temperature (T_(j)) is less than the second criticalvalue (T_(j,Cr2)), processing determines whether the cold plate isoperating at stage 1 962, and if “yes”, issues the “High Coolant Flow”warning 966 and decreases the coolant flow rate 968. Processing thenwaits time interval t 912 before checking whether active control isenabled 900. If the cold plate is not operating at stage 1 962, then thesystem switches the cold plate to stage 1 964, and processing waits timeinterval t 912, before checking whether active control is enabled 900.

FIG. 10 represents the heat transfer benefit that can be obtained byreducing channel cross-sectional area. For laminar internal (channel)flows, in the developed flow regime, the Nusselt number (Nu) isconstant. For constant temperature boundary conditions, Nu=3.66 and forconstant heat flux boundary conditions, Nu=4.36. The Nusselt number canbe expressed as Nu=h×D_(h)/k, where h is the convective heat transfercoefficient, D_(h) is the hydraulic diameter of the channel(s) and k isthe thermal conductivity of the coolant. Let D_(h,1) represent thehydraulic diameter of the taller channel(s) and D_(h,2) represent thehydraulic diameter of the shorter channel(s). Since the taller channelshave larger cross-section for coolant flow, D_(h,1)>D_(h,2). For a fixedcoolant, the thermal conductivity (k) of the coolant can be assumed tobe constant. Thus, the convective heat transfer coefficient in thedeveloped flow regime for the shorter channels (h₂) with respect to thatof the taller channels (h₁) can be estimated using the Nusselt numberrelationship.

Nu=h₁×D_(h,1)/k=h₂×D_(h,2)/k. Thus, h₂=h₁×D_(h,1)/D_(h,2). Thus, forD_(h,1)>D_(h,2), h₂ is greater than h₁ by the inverse ratio of thecorresponding hydraulic diameters.

In general, the channel flow regime can be divided into 2 sections, thatis, a thermally developing flow regime and a thermally developed flowregime. In the thermally developing flow regime the heat transfer ratesare very high, while in the thermally developed region, the heattransfer rates reach a constant value. Thus, by having a longerthermally developing flow region higher heat transfer rates can beobtained. The thermal development length (l_(e)) for laminar flows isgenerally expressed as l_(e)=0.055×Re×Pr×D_(h), where Re is the flowReynolds Number, Pr is the Prandtl number and is generally constant fora given coolant and D_(h) is the hydraulic diameter. The Reynolds Numbercan be expressed as Re=4{dot over (m)}/μP where, {dot over (m)} is thecoolant mass flow rate and μ is the coolant viscosity and P is theperimeter of the channels cross-section. For the same coolant mass flowrate, the Reynolds Number for the shorter channels with respect to thatof the taller channels can be expressed as:

$\frac{{Re}_{2}}{{Re}_{1}} = {\frac{P_{1}}{P_{2}}.}$

Thus, for P₁>P₂, the Re₂ is greater than Re₁ by the inverse ratio of thecorresponding perimeters. Thus, the thermally developing length for theshorter channels (l_(e,2)) with respect to that of the taller (l_(e,1))can be expressed as:

$\frac{l_{e,2}}{l_{e,1}} = {\frac{P_{1}D_{h,2}}{P_{2}D_{h,1}}.}$

It can be mathematically shown that in general l_(e,2)>l_(e,1).

Also, in general, the heat transfer coefficient increases with theReynolds Number (see FIG. 11). Thus by increasing the flow ReynoldsNumber, the coolant flow through the cold-plate could be transitionedfrom laminar to turbulent flow, resulting in higher heat transfer rates.Thus the shorter channels have triple benefit on the heat transferrate—relatively longer thermally developing region, increased heattransfer rates in the thermally developed region and possibility oftransition to turbulent flow. However, this increased benefit comes at aprice of increased pressure drop across the cold plate resulting inincreased coolant pumping power. This increased pumping power is localto the cold-plates and its effect on the overall system (data centerlevel) could possibly be negligible.

As noted, FIGS. 12A-15C depict different embodiments of a dynamicallyvariable, coolant-cooled cold plate, in accordance with one or moreaspects of the present invention.

In FIGS. 12A & 12B, a cooling apparatus 1200 is illustrated comprising acoolant-cooled cold plate 1210 and a controller 1230 for activelycontrolling a physical configuration, and thus a physicalcharacteristic, of the coolant-cooled cold plate. The coolant-cooledcold plate 1210 includes, in this embodiment, a plurality ofparallel-extending, coolant-carrying channels 1211 separated by channelwalls 1212. In this example, the coolant flow cross-section through thecoolant-carrying channels is constant, but the number of channelscarrying coolant flow is varied by adjusting positioning of one or moreadjustable isolation plates 1215 to selectively control the number ofchannels within the cold plate with coolant flow.

In the illustrated embodiment, two isolation plates are shown laterallymovable (or translatable) to reduce the effective number of channels(based, for instance, on the heat being dissipated by the one or moreelectronic components being cooled by then cold plate and the coolantflow rate through the cold plate). The adjustable plates 1215 aremounted on a pair of threaded shafts 1214 that are engaged by one ormore gears 1213 (e.g., in the form of a worm-gear arrangement) toprovide the rotational motion to the shafts 1214. In otherconfigurations, shafts 1214 could be rotatable without using arotational gear 1213, such as illustrated in FIGS. 12A & 12B. Forexample, in an alternate configuration, no gear may be included withinthe cold plate, with a gear arrangement provided outside of the coldplate to synchronously rotate the two shafts.

Coolant flows into the parallel, coolant-carrying channels 1211 via oneor more coolant inlets 1220 and egresses via one or more coolant outlet1221. In the depicted example, the coolant inlet 1220 and coolant outlet1221 are through respective gears 1213, as illustrated. Channel walls1212 are separated from gears 1213 such that a coolant supply space 1222and coolant return space 1223 are provided within the coolant-cooledcold plate 1210. As the gear is rotated, for example, clockwise, due tothe threading of the shaft, the shaft also rotates in the clockwisedirection. This causes the isolation plates 1215 to move forward towardseach other. When the gear is rotated in the counterclockwise direction,the shafts also move counterclockwise, and thus, the isolation platesmove away from each other. The isolation plates may be configured suchthat coolant within the coolant supply space 1222 and coolant returnspace 1223 is constrained by the laterally-translatable isolation plates1215 so that one or more outer coolant-carrying channels 1211 of thecoolant-carrying cold plate become isolated from the coolant flowthrough the cold plate. In other configurations, the movable isolationplates could comprise grooves for allowing a reduced coolant flowthrough the outer channels, notwithstanding translation of the isolationplates towards each other.

FIGS. 12A & 12B depict isolation plates 1215 at a first stage. In thisfirst stage, a maximum number of coolant-carrying channels are carryingcoolant flow through the cold plate. This results in a lower ReynoldsNumber for a same coolant mass flow rate through the cold plate. Thisstage 1 configuration could also be useful where coolant spreadingthrough the cold plate is required.

In FIGS. 12C & 12D, a stage 2 configuration is illustrated, whereinisolation plates 1215 are moved partially towards each other, resultingin coolant flow through a reduced number of coolant-carrying channels1211. This positioning could be useful when some thermal spreading isrequired through the cold plate.

FIGS. 12E & 12F depict a stage 3 for the coolant-cooled cold plate ofFIGS. 12A-12D. At this stage, the movable isolation plates 1215 are at aclosest position relative to each other, allowing the coolant to flowthrough a minimum number of channels. This concentrating of the coolantwithin a reduced number of channels results in a relatively higherReynolds Number for the same coolant mass flow rate through the coldplate. Note that in one embodiment, the processing of FIG. 9 may beimplemented to control positioning of the moveable isolation plates 1215in the coolant-cooled cold plate 1210 embodiment of FIGS. 12A-12F.

FIGS. 13A & 13B depict another embodiment of a cooling apparatus,generally denoted 1300, in accordance with one or more aspects of thepresent invention. Cooling apparatus 1300 includes a coolant-cooled coldplate 1310 and a controller 1330 for actively controlling a physicalconfiguration, and thus a physical characteristic, of the coolant-cooledcold plate. The coolant-cooled cold plate 1310 includes, in thisembodiment, a plurality of parallel-extending, coolant-carrying channels1311 separated by channel walls 1312. A coolant flow cross-sectionthrough a single coolant-carrying channel of the plurality ofcoolant-carrying channels 1311 is determined by a width (‘W’) and aheight (‘H’) of the coolant-carrying channel. As described herein, byadjusting the height ‘H’ of the coolant-carrying channels, the coolantflow cross-section through the channels, as well as through thecoolant-cooled cold plate, can be automatically, dynamicallyreconfigured employing, for example, processing such as described abovein connection with FIG. 9. This dynamic reconfiguration of the height ofthe coolant-carrying channels in the example of FIGS. 13A & 13B is anexample of a physical reconfiguration of a cold plate which is achievedby dynamically adjusting one or more plates within the cold plate, inaccordance with one or more aspects of the present invention.

In the depicted embodiment, an adjustable mid-plate 1315 is provided,which is controlled by controller 1330 to adjust the height ‘H’ of theparallel, coolant-carrying channels 1311 relative to a base plate 1313of the coolant-cooled cold plate 1310. In this example, adjustablemid-plate 1315 is magnetically adjustable, with permanent magnets 1316being affixed to or integrated within adjustable mid-plate 1315, andcontrollable magnets 1317, such as solenoid magnets, associated with anupper plate 1318 of coolant-cooled plate 1310. Bellows 1301 (or springs)may be provided around the portion of solenoid magnets 1317 extendinginto coolant-cooled cold plate 1310 to couple adjustable mid-plate 1315to upper plate 1318. By controlling the magnetic force applied bycontrollable magnets 1317, including polarity of the magnets, theadjustable mid-plate 1315 can be moved to adjust the effective channelheight ‘H’ of the plurality of parallel, coolant-carrying channels 1311.Hard stops 1314 may be provided, for example, at a desired minimumchannel height ‘H’.

Coolant flows into the parallel, coolant-carrying channels 1311 via oneor more coolant inlets 1320, and egresses via one or more coolantoutlets 1321. Note that the space 1319 between adjustable mid-plate 1315and upper plate 1318 of the coolant-cooled cold plate may be filled withcoolant, but that coolant flow is principally through coolant-carryingchannels 1311.

Controller 1330 may be within the common electronic system containingthe coolant-cooled cold plate 1310, or could be disposed within anotherelectronic system within the same electronics rack, or even remote fromthe rack containing the coolant-cooled cold plate. The controllerimplements a control process for automatically controlling position ofadjustable mid-plate 1315, and thus control (in this example) thecoolant flow cross-section through the plurality of coolant-carryingchannels, and therefore through the cold plate. The controller 1330actively controls the physical configuration of the coolant-cooled coldplate, and this active control is to, for instance, optimally cool theassociated electronic components being cooled by the cold plate, whileat the same time reducing power consumption of the cooling apparatus. Inone embodiment, the adjustable mid-plate is adjusted by the controllerto reconfigure the effective channel height based on, for example, heatdissipated by the associated electronic component(s) being cooled and/orthe coolant flow rate through the coolant-cooled cold plate, forexample, in a manner similar to that described above in connection withFIG. 9.

The adjustable mid-plate 1315 may contain grooves or slots correspondingto each channel wall 1312 for movement of the plate normal to theparallel-extending, coolant-carrying channel lengths. As noted, polarityof the adjustable magnets 1317 associated with upper plate 1318 of thecold plate can be toggled, and magnetic strength can be varied, toattract or repel the adjustable mid-plate, as desired. Thus, bycontrolling the magnetic force between the adjustable mid-plate and theupper plate, the effective channel height can be adjusted andcontrolled. Polarity of the controllable magnets 1317 can be toggledbased on the direction of electrical current through the windings of thesolenoid magnets. The magnetic strength of the solenoid magnets may becontrolled based on the magnitude of electrical current through thewindings. The multiple bellows 1301 could alternatively comprisemultiple springs. These structures facilitate coupling of the adjustablemid-plate to the upper plate.

FIGS. 13C-13E depict three different stages (or positions) of theadjustable mid-plate 1315 relative to, for example, base plate 1313 ofthe coolant-cooled cold plate. At stage 1, illustrated in FIG. 13C, themagnetic components on upper plate 1318 have polarity opposite to thatof the magnets on the adjustable mid-plate 1315, so that the platesattract each other and attach together. In this position, the height ‘H’of the coolant-carrying channels is at a maximum, and the largercross-sectional flow areas result in a relatively lower Reynolds Numberfor the same coolant mass flow rate, that is, compared with thepositioning of the adjustable mid-plate in either stage 2 or stage 3illustrated in FIGS. 13D & 13E, respectively.

As illustrated in FIG. 13D, at stage 2, the magnetic components of theupper plate 1318 could have similar polarity or partially similar andpartially opposite polarity, to that of the movable mid-plate 1315. Themagnetic strength of the magnetic components on the upper plate could bevaried as well so that the adjustable mid-plate stays at an intermediateposition, such as illustrated in the figure. At stage 3, illustrated inFIG. 13E, the magnetic components on the upper plate have similarpolarity as that on the adjustable mid-plate, so that the plates repeleach other and, in this example, the mid-plate rests on hard stops 1314of the coolant-cooled cold plate. The magnetic strength of the magneticcomponents on the upper plate may be adjusted so that the adjustablemid-plate is farthest from the top plate, resulting in the smallestpossible effective channel height ‘H’.

Note that, at stage 1, the channels are the highest, resulting in alarge cross-sectional area for coolant flow, while at stage 3, thechannels have the smallest height, resulting in a smallercross-sectional area for coolant flow. Thus, for a given mass flow rate,the Reynolds Number for the coolant flow is highest in stage 3. At stage2, the channel height is relatively smaller as compared to stage 1,resulting in a relatively smaller cross-sectional flow rate. Thus, for agiven mass flow rate, the Reynolds Number for the coolant flow isrelatively higher in stage 2 than for stage 1. For a given mass flowrate through the cold plate, stage 3 offers the highest heat transferrate. It should be noted that the convective heat transfer rates arehigher for larger Reynolds Number flows.

FIGS. 14A & 14B depict a further embodiment of a cooling apparatus 1400,in accordance with one or more aspects of the present invention. Coolingapparatus 1400 includes a coolant-cooled cold plate 1410 and acontroller 1430 for actively controlling a physical configuration of thecoolant-cooled cold plate. In this implementation, the coolant-cooledcold plate 1410 includes an adjustable jet orifice plate 1415 with aplurality of orifices 1413. A height ‘H’ of adjustable jet orifice plate1415 is adjustable by controller 1430 to adjust the impingement heightof the jet flows from jet orifices 1413 onto a target surface 1411, forexample, of coolant-cooled cold plate 1410. In one embodiment,coolant-cooled cold plate 1410 sets on top of one or more electroniccomponents to be cooled. The orifices 1413 in adjustable jet orificeplate 1415 may be arrayed in a regular pattern, as illustrated in FIG.14B, or alternatively, may be irregularly-spaced depending, for example,on the location of the underlying one or more heat-generating electroniccomponents being cooled by the cold plate.

In the depicted embodiment, the adjustable jet orifice plate 1415 ismagnetically adjustable, and comprises permanent magnets 1416 affixed toor integrated with the adjustable jet orifice plate. Controllablemagnets 1417, such as solenoid magnets, are associated with upper plate1418 of coolant-cooled cold plate 1410. By controlling the magneticforce applied by the controllable magnets 1417, including polarity ofthe magnets, the adjustable jet orifice plate 1415 can be moved toadjust the effective impingement height ‘H’ of the jet orifice platerelative to target surface 1411. In this implementation, the adjustableimpingement height is a physical characteristic of the coolant-cooledcold plate that is changeable by control of the magnetic force beingapplied to the adjustable jet orifice plate 1415. Hard stops 1414 areprovided, for example, at a desired minimum jet impingement height ‘H’.

Coolant flows into the coolant-cooled cold plate via one or more coolantinlets 1420, and after impinging on the target surface 1411, egressesvia one or more coolant outlets 1421. Note that space 1419 between theadjustable jet orifice plate 1415 and upper plate 1418 operates as acoolant supply plenum in this example, and that adjustable bellows 1401are provided around the magnet portions of magnets 1416, 1417 within thespace 1419 to, for example, couple the adjustable jet orifice plate tothe upper plate. The adjustable jet orifice plate could be attached tothe upper plate 1480 via either the multiple bellows 1401, oralternatively, multiple springs. Further, the controllable magneticcomponents 1417 on the upper plate could be solenoid magnets whosepolarity could be toggled based on the direction of electrical currentthrough the windings. Also, the magnetic strength of the solenoidmagnets could be controlled based on the magnitude of electrical currentthrough the windings.

Controller 1430 may be within a common electronic system containing thecoolant-cold plate 1410, or could be disposed in another electronicsystem within the same electronics rack, or even remote from theelectronics rack containing the coolant-cooled cold plate. Thecontroller implements a control process for automatically controllingposition of the adjustable jet orifice plate 1415, and thus controllingthe impingement height of the jet flow directed towards the targetsurface 1411 being cooled. Thus, the controller 1430 actively controls aconfiguration and a physical characteristic of the coolant-cooled coldplate, and this active control is to, for instance, optimally cool theassociated electronic components being cooled by the cold plate, whileat the same time, reduce cooling power consumption. In one embodiment,the adjustable jet orifice plate is adjusted by the control toreconfigure the effective jet impingement height based on, for example,heat dissipated by the associated electronic component(s) being cooled,and/or the coolant flow rate through the coolant-cooled cold plate.

FIGS. 14C-14E depict three different stages of the adjustable jetorifice plate 1415 within the coolant-cooled cold plate 1410 of FIGS.14A & 14B. Processing for controlling the position of the adjustableplate between these three stages may be similar to that described abovein connection with FIG. 9. At stage 1, illustrated in FIG. 14C, themagnetic components on the upper plate have polarity opposite to that ofthe permanent magnets on the adjustable jet orifice plate 1415, so thatthe adjustable plate is attracted to the upper plate, and the two platesattach, as illustrated. Depending on the flow rate, this positioningcould provide optimum impingement height and/or separation. At stage 2,illustrated in FIG. 14D, the magnetic components on upper plate 1418 mayhave similar polarity, or partially similar and partially oppositepolarity, as the polarity of the permanent magnets on the adjustableorifice plate 1415. The magnetic strength of magnetic components 1417 onupper plate 1418 may be varied as well, so that the moveable plate 1415stays at an intermediate position, such as illustrated in FIG. 14D. Atstage 3, illustrated in FIG. 14E, the magnetic components 1417 on upperplate 1418 have similar polarity to the permanent magnets on adjustablejet orifice plate 1415, such that the plates repel each other. Themagnetic strength of the magnetic components 1417 on upper plate 1418can be adjusted so that the adjustable jet orifice plate 1415 isfarthest from the upper plate, resulting in a smallest effectiveimpingement height ‘H’. Depending on the flow rate, any one of the threestages could provide the optimum impingement height/separation in orderto optimize cooling performance of the cold plate.

FIGS. 15A-15C depict a further embodiment of a cooling apparatus,generally denoted 1500, in accordance with one or more aspects of thepresent invention. Cooling apparatus 1500 includes a coolant-cooled coldplate 1510 and a controller 1530 for actively controlling a physicalconfiguration, and thus a physical characteristic, of the coolant-cooledcold plate. The coolant-cooled cold plate 1510 includes, in thisembodiment, one or more coolant-carrying channels 1511 defined between abase plate 1512 and an adjustable plate 1515. In one example, adjustableplate 1515 is a piezoelectric plate, which may be adjusted to adjust theeffective channel height between the adjustable plate and base plate1512 of the cold plate based, for example, on the heat dissipated by theelectronic component(s) being cooled and/or the coolant flow ratethrough the cold plate. There may either be a single adjustable plate1515 for the entire cold plate, or (for example) there may be anadjustable plate for each individual coolant-carrying channel 1511 ofthe cold plate. In the case of a single adjustable plate 1515 for theentire cold plate, the plate may contain grooves/slots corresponding toeach channel wall (not shown) for movement of the adjustable platenormal to the length of the channels. The adjustable plate may be madeout of a piezoelectric material, such that by controlling the directionand magnitude of electrical current through the plate, the plate couldbe bent to alter the effective channel height. Thus, the heat transferrates near the center of the channels (or inside the channels, ingeneral) can be increased or decreased to optimally cool the electroniccomponent(s).

At stage 1, illustrated in FIG. 15A, a voltage/current is applied to theadjustable plate such that the plate deflects away from base plate 1512of the coolant-carrying channel(s) 1511, resulting in a relativelylarger cross-section for coolant flow. At stage 2, illustrated in FIG.15B, no voltage/current is applied across the adjustable piezoelectricplate, and thus, the plate stays in a neutral position, resulting in arelatively smaller cross-sectional area for coolant flow. At stage 3,illustrated in FIG. 15C, a voltage/current is applied so that the platedeflects towards the base plate 1512, resulting in a relatively smallercross-section for coolant flow. This also results in flow accelerationin the region between the cold plate inlet and channel center, resultingin increased heat transfer rates in that region. At stage 1, thechannels are the tallest, resulting in a larger cross-section forcoolant flow. Thus, for a given mass flow rate, the Reynolds Number forthe flow in stage 1 (FIG. 15A) is the smallest of the three stages,while at stage 3, the channels are the smallest, resulting in a smallercross-section for coolant flow. Thus, for a given mass flow rate, theReynolds Number for the coolant flow is the highest at stage 3. At stage2, the channel height is relatively smaller than stage 1, resulting in arelatively smaller cross-section for coolant flow, and for a given massflow rate, a larger Reynolds Number, compared with stage 1. For a givenmass flow rate through the cold plate, stage 3 offers the highest heattransfer rates. It should be noted that the convective heat transferrates are higher for higher Reynolds Number flows.

Implementation of the cooling apparatuses and cooling methods describedabove may require replacement of various hardware components, or use ofadditional hardware components, within an electronics rack. Once therequired hardware is installed, controls associated with each cold platecould be implemented in a number of ways. One way is to program thecontrol process onto a programmable logic controller (PLC) unit of thedata center facility. Another way is to implement the control processingon a remote computer/server which takes in the required inputinformation from the data center components and outputs the optimumoperational points for the various coolant-cooled cold plates within thesystem. Another approach to implementing the control depends on wherethe components are installed. For instance, if the components arelocated inside of each server in a multi-server rack, then the controlcould be programmed into that server's management controller, such asthe baseboard management controller (BMC), intelligent platformmanagement interface (IPMI), etc.

As will be appreciated by one skilled in the art, one or more aspects ofthe present invention may be embodied as a system, method or computerprogram product. Accordingly, one or more aspects of the presentinvention may take the form of an entirely hardware embodiment, anentirely software embodiment (including firmware, resident software,micro-code, etc.) or an embodiment combining software and hardwareaspects that may all generally be referred to herein as a “circuit,”“module” or “system”. Furthermore, one or more aspects of the presentinvention may take the form of a computer program product embodied inone or more computer readable medium(s) having computer readable programcode embodied thereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readablestorage medium. A computer readable storage medium may be, for example,but not limited to, an electronic, magnetic, optical, electromagnetic,infrared or semiconductor system, apparatus, or device, or any suitablecombination of the foregoing. More specific examples (a non-exhaustivelist) of the computer readable storage medium include the following: anelectrical connection having one or more wires, a portable computerdiskette, a hard disk, a random access memory (RAM), a read-only memory(ROM), an erasable programmable read-only memory (EPROM or Flashmemory), an optical fiber, a portable compact disc read-only memory(CD-ROM), an optical storage device, a magnetic storage device, or anysuitable combination of the foregoing. In the context of this document,a computer readable storage medium may be any tangible medium that cancontain or store a program for use by or in connection with aninstruction execution system, apparatus, or device.

Referring now to FIG. 16, in one example, a computer program product1600 includes, for instance, one or more non-transitory computerreadable storage media 1602 to store computer readable program codemeans or logic 1604 thereon to provide and facilitate one or moreaspects of the present invention.

Program code embodied on a computer readable medium may be transmittedusing an appropriate medium, including but not limited to, wireless,wireline, optical fiber cable, RF, etc., or any suitable combination ofthe foregoing.

Computer program code for carrying out operations for one or moreaspects of the present invention may be written in any combination ofone or more programming languages, including an object orientedprogramming language, such as Java, Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language, assembler or similar programming languages. Theprogram code may execute entirely on the user's computer, partly on theuser's computer, as a stand-alone software package, partly on the user'scomputer and partly on a remote computer or entirely on the remotecomputer or server. In the latter scenario, the remote computer may beconnected to the user's computer through any type of network, includinga local area network (LAN) or a wide area network (WAN), or theconnection may be made to an external computer (for example, through theInternet using an Internet Service Provider).

One or more aspects of the present invention are described herein withreference to flowchart illustrations and/or block diagrams of methods,apparatus (systems) and computer program products according toembodiments of the invention. It will be understood that each block ofthe flowchart illustrations and/or block diagrams, and combinations ofblocks in the flowchart illustrations and/or block diagrams, can beimplemented by computer program instructions. These computer programinstructions may be provided to a processor of a general purposecomputer, special purpose computer, or other programmable dataprocessing apparatus to produce a machine, such that the instructions,which execute via the processor of the computer or other programmabledata processing apparatus, create means for implementing thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

The flowchart and block diagrams in the figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of one or more aspects of the present invention. In thisregard, each block in the flowchart or block diagrams may represent amodule, segment, or portion of code, which comprises one or moreexecutable instructions for implementing the specified logicalfunction(s). It should also be noted that, in some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts, or combinations of special purpose hardware andcomputer instructions.

In addition to the above, one or more aspects of the present inventionmay be provided, offered, deployed, managed, serviced, etc. by a serviceprovider who offers management of customer environments. For instance,the service provider can create, maintain, support, etc. computer codeand/or a computer infrastructure that performs one or more aspects ofthe present invention for one or more customers. In return, the serviceprovider may receive payment from the customer under a subscriptionand/or fee agreement, as examples. Additionally or alternatively, theservice provider may receive payment from the sale of advertisingcontent to one or more third parties.

In one aspect of the present invention, an application may be deployedfor performing one or more aspects of the present invention. As oneexample, the deploying of an application comprises providing computerinfrastructure operable to perform one or more aspects of the presentinvention.

As a further aspect of the present invention, a computing infrastructuremay be deployed comprising integrating computer readable code into acomputing system, in which the code in combination with the computingsystem is capable of performing one or more aspects of the presentinvention.

As yet a further aspect of the present invention, a process forintegrating computing infrastructure comprising integrating computerreadable code into a computer system may be provided. The computersystem comprises a computer readable medium, in which the computermedium comprises one or more aspects of the present invention. The codein combination with the computer system is capable of performing one ormore aspects of the present invention.

Although various embodiments are described above, these are onlyexamples. Further, other types of computing environments can benefitfrom one or more aspects of the present invention.

As a further example, a data processing system suitable for storingand/or executing program code is usable that includes at least oneprocessor coupled directly or indirectly to memory elements through asystem bus. The memory elements include, for instance, local memoryemployed during actual execution of the program code, bulk storage, andcache memory which provide temporary storage of at least some programcode in order to reduce the number of times code must be retrieved frombulk storage during execution.

Input/Output or I/O devices (including, but not limited to, keyboards,displays, pointing devices, DASD, tape, CDs, DVDs, thumb drives andother memory media, etc.) can be coupled to the system either directlyor through intervening I/O controllers. Network adapters may also becoupled to the system to enable the data processing system to becomecoupled to other data processing systems or remote printers or storagedevices through intervening private or public networks. Modems, cablemodems, and Ethernet cards are just a few of the available types ofnetwork adapters.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprise” (andany form of comprise, such as “comprises” and “comprising”), “have” (andany form of have, such as “has” and “having”), “include” (and any formof include, such as “includes” and “including”), and “contain” (and anyform contain, such as “contains” and “containing”) are open-endedlinking verbs. As a result, a method or device that “comprises”, “has”,“includes” or “contains” one or more steps or elements possesses thoseone or more steps or elements, but is not limited to possessing onlythose one or more steps or elements. Likewise, a step of a method or anelement of a device that “comprises”, “has”, “includes” or “contains”one or more features possesses those one or more features, but is notlimited to possessing only those one or more features. Furthermore, adevice or structure that is configured in a certain way is configured inat least that way, but may also be configured in ways that are notlisted.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements in the claims below, if any, areintended to include any structure, material, or act for performing thefunction in combination with other claimed elements as specificallyclaimed. The description of the present invention has been presented forpurposes of illustration and description, but is not intended to beexhaustive or limited to the invention in the form disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the invention.The embodiment was chosen and described in order to explain theprinciples of the invention and the practical application, and to enableothers of ordinary skill in the art to understand the invention throughvarious embodiments and the various modifications thereto which aredependent on the particular use contemplated.

What is claimed is:
 1. A method comprising: monitoring a variableassociated with at least one of a coolant-cooled cold plate or anelectronic component being cooled by the coolant-cooled cold plate; anddynamically varying, based on the monitored variable, a physicalconfiguration of the coolant-cooled cold plate, wherein the dynamicallyvarying the physical configuration alters at least one of thermal orfluid dynamic performance of the coolant-cooled cold plate cooling theelectronic component.
 2. The method of claim 1, wherein thecoolant-cooled cold plate comprises at least one coolant-carryingchannel, and the dynamically varying comprises dynamically reconfiguringa coolant flow cross-section through the coolant-cooled cold plate basedon the monitored variable.
 3. The method of claim 2, wherein thedynamically varying comprises switching the physical configuration ofthe coolant-cooled cold plate among multiple predefined stages based onthe monitored variable, the multiple predefined stages definingdifferent cross-sectional coolant flow areas through the at least onecoolant-carrying channel of the coolant-cooled cold plate.
 4. The methodof claim 2, wherein varying the physical configuration varies a physicalcharacteristic of the coolant-cooled cold plate, the physicalcharacteristic comprising a height of the at least one coolant-carryingchannel of the coolant-cooled cold plate.
 5. The method of claim 4,wherein the coolant-cooled cold plate further comprises an adjustablemid-plate, wherein adjustment of the adjustable mid-plate varies theheight of the at least one coolant-carrying channel of thecoolant-cooled cold plate.
 6. The method of claim 5, wherein theadjustable mid-plate is magnetically adjustable, and wherein thedynamically varying the physical configuration comprises magnetically,actively controlling the position of the mid-plate within thecoolant-cooled cold plate to control the height of the at least onecoolant-carrying channel of the coolant-cooled cold plate.
 7. The methodof claim 6, wherein the adjustable mid-plate comprises a plurality ofpermanent magnets coupled thereto.
 8. The method of claim 6, wherein thedynamically varying comprises actively controlling a magnetic forceapplied to the mid-plate to control positioning of the adjustablemid-plate within the coolant-cooled cold plate.
 9. The method of claim6, further comprising a plurality of solenoid-based magnets associatedwith the coolant-cooled cold plate, and wherein the dynamically varyingcomprises actively controlling a magnetic force applied to theadjustable mid-plate to control positioning of the adjustable mid-platewithin the coolant-cooled cold plate.
 10. The method of claim 1, whereinthe coolant-cooled cold plate comprises a plurality of coolant-carryingchannels extending at least partially in parallel, and wherein thedynamically varying the physical configuration of the coolant-cooledcold plate comprises dynamically reconfiguring a coolant flowcross-section through the plurality of coolant-carrying channels of thecoolant-cooled cold plate.
 11. The method of claim 10, wherein thedynamically varying comprises dynamically reconfiguring whichcoolant-carrying channels of the plurality of parallel, coolant-carryingchannels within the coolant-cooled cold plate have coolant flowingtherethrough.
 12. The method of claim 10, wherein the dynamicallyvarying comprises actively adjusting positioning of at least oneisolation plate within the coolant-cooled cold plate to selectivelyisolate one or more coolant-carrying channels of the plurality ofcoolant-carrying channels from coolant flowing through thecoolant-cooled cold plate.
 13. The method of claim 12, wherein thedynamically varying comprises dynamically controlling positioning of theat least one laterally-adjustable isolation plate between multipledefined plate position stages to selectively alter the at least onethermal or fluid dynamic performance of the coolant-cooled cold platebased on the monitored variable.
 14. The method of claim 1, wherein thecoolant-cooled cold plate comprises at least one coolant-carryingchannel, and a piezoelectric plate associated with the at least onecoolant-carrying channel, and wherein the dynamically varying comprisesactively controlling the piezoelectric plate to dynamically reconfigurea coolant flow cross-section through the at least one coolant-carryingchannel, and thereby alter the at least one of thermal or fluid dynamicperformance of the coolant-carrying cold plate.
 15. The method of claim1, wherein the coolant-cooled cold plate further comprises an adjustablejet orifice plate with a plurality of jet orifices therein, and whereinthe dynamically varying comprises actively controlling an effectiveimpingement height between the adjustable jet orifice plate and a targetsurface, wherein varying the physical configuration varies a physicalcharacteristic of the coolant-cooled cold plate, the physicalcharacteristic comprising a height of the adjustable jet orifice platerelative to the target surface.
 16. A method comprising: monitoring avariable associated with at least one of a coolant-cooled cold plate oran electronic component being cooled by the coolant-cooled cold plate;and dynamically varying, based on the monitored variable, a physicalconfiguration of the coolant-cooled cold plate, the dynamically varyingcomprising dynamically reconfiguring the physical configuration of thecoolant-cooled cold plate by automatically adjusting at least oneadjustable plate within the coolant-cooled cold plate, wherein thedynamically varying the physical configuration alters at least one ofthermal or fluid dynamic performance of the coolant-cooled cold platecooling the electronic component.
 17. The method of claim 16, whereinthe coolant-cooled cold plate comprises at least one coolant-carryingchannel, and the dynamically varying comprises dynamically reconfiguringa coolant flow cross-section through the coolant-cooled cold plate,based on the monitored variable, by automatically adjusting the at leastone adjustable plate within the coolant-cooled cold plate.
 18. Themethod of claim 17, wherein the dynamically varying comprises activelyswitching the physical configuration of the coolant-cooled cold plateamong multiple predefined stages based on the monitored variable, themultiple predefined stages defining different cross-sectional coolantflow areas through the at least one coolant-carrying channel of thecoolant-cooled cold plate.
 19. The method of claim 16, wherein thecoolant-cooled cold plate further comprises an adjustable jet orificeplate with a plurality of jet orifices therein, and wherein thedynamically varying comprises actively controlling an effectiveimpingement height between the adjustable jet orifice plate and a targetsurface, wherein varying the physical configuration varies a physicalcharacteristic of the coolant-cooled cold plate, the physicalcharacteristic comprising a height of the adjustable jet orifice platerelative to the target surface.
 20. A method comprising: providing acoolant-cooled cold plate configured to couple to at least oneelectronic component to be cooled, the coolant-cooled cold platecomprising at least one coolant-carrying channel, and at least oneadjustable plate for adjusting a physical configuration of thecoolant-cooled cold plate, wherein adjustment of the at least oneadjustable plate reconfigures the coolant-cooled cold plate and altersat least one of thermal or fluid dynamic performance of thecoolant-cooled cold plate; and providing a controller to activelycontrol positioning of the at last one adjustable plate within thecoolant-cooled cold plate based on a monitored variable associated withat least one of the coolant-cooled cold plate or the at least oneelectronic component to be cooled by the coolant-cooled cold plate.